{
 "cells": [
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "# 检索器(召回器) Retrievers\n",
    "## 介绍\n",
    "向量数据库本身已经包含了实现召回功能的函数方法( `similarity_search` )。该函数通过 **计算原始查询向量与数据库中存储向量之间的相似度来实现召回**。\n",
    "\n",
    "LangChain还提供了 **更加复杂的召回策略** ，这些策略被集成在Retrievers（检索器或召回器）组件中。\n",
    "\n",
    "Retrievers（检索器）是一种用于从大量文档中检索与给定查询相关的文档或信息片段的工具。检索器 **不需要存储文档** ，只需要 **返回（或检索）文档** 即可。"
   ],
   "id": "9b887b4fea351b89"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## 文档嵌入模型",
   "id": "9ed21bc58b21225"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T03:00:33.042945Z",
     "start_time": "2025-11-26T03:00:31.725092Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import os\n",
    "\n",
    "import dotenv\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "dotenv.load_dotenv(override=True)\n",
    "os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\")\n",
    "os.environ[\"OPENAI_BASE_URL\"] = os.getenv(\"OPENAI_BASE_URL\")\n",
    "\n",
    "embeddings_model = OpenAIEmbeddings(model=\"text-embedding-ada-002\")"
   ],
   "id": "660db00fae01ea4d",
   "outputs": [],
   "execution_count": 1
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "> 举例1：使用检索器实现简单的召回功能",
   "id": "6dff188fbeccd4d4"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T02:44:53.296022Z",
     "start_time": "2025-11-26T02:44:51.982171Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_community.document_loaders import TextLoader\n",
    "from langchain_community.vectorstores import FAISS\n",
    "from langchain_text_splitters import CharacterTextSplitter\n",
    "\n",
    "# 加载文档\n",
    "print(\"开始加载文档..........\")\n",
    "text_loader = TextLoader(\"asset/load/09-ai1.txt\", encoding=\"utf-8\")\n",
    "source = text_loader.load()\n",
    "\n",
    "# 拆分文档\n",
    "print(\"开始拆分文档..........\")\n",
    "text_splitter = CharacterTextSplitter(\n",
    "    separator=\"\\n\\n\",\n",
    "    chunk_size=1000,\n",
    "    chunk_overlap=100,\n",
    ")\n",
    "chunks = text_splitter.split_documents(source)\n",
    "print(len(chunks))\n",
    "\n",
    "print(\"开始创建向量数据库..........\")\n",
    "db = FAISS.from_documents(chunks, embeddings_model)\n",
    "\n",
    "print(\"开始创建检索器..........\")\n",
    "retriever = db.as_retriever()\n",
    "\n",
    "print(\"开始召回文档..........\")\n",
    "respDoc = retriever.invoke(\"深度学习是什么？\")\n",
    "print(len(respDoc))\n",
    "for doc in respDoc:\n",
    "    print(f\"--->>>>>>>metadate:{doc.metadata}, content:\\n{doc.page_content}\\n\")"
   ],
   "id": "dc40078da2767a36",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "开始加载文档..........\n",
      "开始拆分文档..........\n",
      "3\n",
      "开始创建向量数据库..........\n",
      "开始创建检索器..........\n",
      "开始召回文档..........\n",
      "3\n",
      "--->>>>>>>metadate:{'source': 'asset/load/09-ai1.txt'}, content:\n",
      "3. 人工智能的核心技术\n",
      "3.1 机器学习\n",
      "机器学习是人工智能的核心技术之一，通过算法使计算机从数据中学习并做出决策。常见的机器学习算法包括监督学习、无监督学习和强化学习。监督学习通过标记数据进行训练，无监督学习则从未标记数据中寻找模式，强化学习则通过与环境交互来优化决策。\n",
      "3.2 深度学习\n",
      "深度学习是机器学习的一个子领域，通过多层神经网络进行特征提取和模式识别。深度学习在图像识别、自然语言处理、语音识别等领域取得了显著成果。常见的深度学习模型包括卷积神经网络（CNN）、循环神经网络（RNN）和长短期记忆网络（LSTM）。\n",
      "3.3 自然语言处理\n",
      "自然语言处理（NLP）是人工智能的一个重要分支，致力于使计算机能够理解和生成人类语言。NLP技术广泛应用于机器翻译、情感分析、文本分类等领域。近年来，基于深度学习的NLP模型（如BERT、GPT）在语言理解任务中取得了突破性进展。\n",
      "3.4 计算机视觉\n",
      "计算机视觉是人工智能的另一个重要分支，致力于使计算机能够理解和处理图像和视频。计算机视觉技术广泛应用于图像识别、目标检测、人脸识别等领域。深度学习模型（如CNN）在计算机视觉任务中取得了显著成果。\n",
      "\n",
      "4. 人工智能的应用领域\n",
      "4.1 医疗健康\n",
      "人工智能在医疗健康领域的应用包括疾病诊断、药物研发、个性化医疗等。通过分析医学影像和患者数据，人工智能可以帮助医生更准确地诊断疾病，提高治疗效果。\n",
      "4.2 金融\n",
      "人工智能在金融领域的应用包括风险评估、欺诈检测、算法交易等。通过分析市场数据和交易记录，人工智能可以帮助金融机构做出更明智的决策，提高运营效率。\n",
      "4.3 教育\n",
      "人工智能在教育领域的应用包括个性化学习、智能辅导、自动评分等。通过分析学生的学习数据，人工智能可以为学生提供个性化的学习建议，提高学习效果。\n",
      "4.4 交通\n",
      "人工智能在交通领域的应用包括自动驾驶、交通管理、智能导航等。通过分析交通数据和路况信息，人工智能可以帮助优化交通流量，提高交通安全。\n",
      "\n",
      "--->>>>>>>metadate:{'source': 'asset/load/09-ai1.txt'}, content:\n",
      "人工智能综述：发展、应用与未来展望\n",
      "\n",
      "摘要\n",
      "人工智能（Artificial Intelligence，AI）作为计算机科学的一个重要分支，近年来取得了突飞猛进的发展。本文综述了人工智能的发展历程、核心技术、应用领域以及未来发展趋势。通过对人工智能的定义、历史背景、主要技术（如机器学习、深度学习、自然语言处理等）的详细介绍，探讨了人工智能在医疗、金融、教育、交通等领域的应用，并分析了人工智能发展过程中面临的挑战与机遇。最后，本文对人工智能的未来发展进行了展望，提出了可能的突破方向。\n",
      "\n",
      "1. 引言\n",
      "人工智能是指通过计算机程序模拟人类智能的一门科学。自20世纪50年代诞生以来，人工智能经历了多次起伏，近年来随着计算能力的提升和大数据的普及，人工智能技术取得了显著的进展。人工智能的应用已经渗透到日常生活的方方面面，从智能手机的语音助手到自动驾驶汽车，从医疗诊断到金融分析，人工智能正在改变着人类社会的运行方式。\n",
      "\n",
      "2. 人工智能的发展历程\n",
      "2.1 早期发展\n",
      "人工智能的概念最早可以追溯到20世纪50年代。1956年，达特茅斯会议（Dartmouth Conference）被认为是人工智能研究的正式开端。在随后的几十年里，人工智能研究经历了多次高潮与低谷。早期的研究主要集中在符号逻辑和专家系统上，但由于计算能力的限制和算法的不足，进展缓慢。\n",
      "2.2 机器学习的兴起\n",
      "20世纪90年代，随着统计学习方法的引入，机器学习逐渐成为人工智能研究的主流。支持向量机（SVM）、决策树、随机森林等算法在分类和回归任务中取得了良好的效果。这一时期，机器学习开始应用于数据挖掘、模式识别等领域。\n",
      "2.3 深度学习的突破\n",
      "2012年，深度学习在图像识别领域取得了突破性进展，标志着人工智能进入了一个新的阶段。深度学习通过多层神经网络模拟人脑的工作方式，能够自动提取特征并进行复杂的模式识别。卷积神经网络（CNN）、循环神经网络（RNN）和长短期记忆网络（LSTM）等深度学习模型在图像处理、自然语言处理、语音识别等领域取得了显著成果。\n",
      "\n",
      "--->>>>>>>metadate:{'source': 'asset/load/09-ai1.txt'}, content:\n",
      "5. 人工智能的挑战与机遇\n",
      "5.1 挑战\n",
      "人工智能发展过程中面临的主要挑战包括数据隐私、算法偏见、安全性问题等。数据隐私问题涉及到个人数据的收集和使用，算法偏见问题则涉及到算法的公平性和透明度，安全性问题则涉及到人工智能系统的可靠性和稳定性。\n",
      "5.2 机遇\n",
      "尽管面临挑战，人工智能的发展也带来了巨大的机遇。人工智能技术的进步将推动各行各业的创新，提高生产效率，改善生活质量。未来，人工智能有望在更多领域取得突破，为人类社会带来更多的便利和福祉。\n",
      "\n",
      "6. 未来展望\n",
      "6.1 技术突破\n",
      "未来，人工智能技术有望在以下几个方面取得突破：一是算法的优化和创新，提高模型的效率和准确性；二是计算能力的提升，支持更复杂的模型和更大规模的数据处理；三是跨学科研究的深入，推动人工智能与其他领域的融合。\n",
      "6.2 应用拓展\n",
      "随着技术的进步，人工智能的应用领域将进一步拓展。未来，人工智能有望在更多领域发挥重要作用，如环境保护、能源管理、智能制造等。人工智能将成为推动社会进步的重要力量。\n",
      "\n",
      "7. 结论\n",
      "人工智能作为一门快速发展的科学，正在改变着人类社会的运行方式。通过不断的技术创新和应用拓展，人工智能将为人类社会带来更多的便利和福祉。然而，人工智能的发展也面临着诸多挑战，需要社会各界共同努力，推动人工智能的健康发展。\n",
      "\n"
     ]
    }
   ],
   "execution_count": 7
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## 使用相关检索策略",
   "id": "d0c6af0860844b87"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 前置代码",
   "id": "c56af4821e6ae94c"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T02:44:53.979674Z",
     "start_time": "2025-11-26T02:44:53.364012Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_community.vectorstores import FAISS\n",
    "from langchain_core.documents import Document\n",
    "\n",
    "# 定义文档\n",
    "document_1 = Document(\n",
    "    page_content=\"经济复苏：美国经济正在从疫情中强劲复苏，失业率降至历史低点。！\",\n",
    ")\n",
    "document_2 = Document(\n",
    "    page_content=\"基础设施：政府将投资1万亿美元用于修复道路、桥梁和宽带网络。\",\n",
    ")\n",
    "document_3 = Document(\n",
    "    page_content=\"气候变化：承诺到2030年将温室气体排放量减少50%。\",\n",
    ")\n",
    "document_4 = Document(\n",
    "    page_content=\" 医疗保健：降低处方药价格，扩大医疗保险覆盖范围。\",\n",
    ")\n",
    "document_5 = Document(\n",
    "    page_content=\"教育：提供免费的社区大学教育。。\",\n",
    ")\n",
    "document_6 = Document(\n",
    "    page_content=\"科技：增加对半导体产业的投资以减少对外国供应链的依赖。。\",\n",
    ")\n",
    "document_7 = Document(\n",
    "    page_content=\"外交政策：继续支持乌克兰对抗俄罗斯的侵略。\",\n",
    ")\n",
    "document_8 = Document(\n",
    "    page_content=\"枪支管制：呼吁国会通过更严格的枪支管制法律。\",\n",
    ")\n",
    "document_9 = Document(\n",
    "    page_content=\"移民改革：提出全面的移民改革方案。\",\n",
    ")\n",
    "document_10 = Document(\n",
    "    page_content=\"社会正义：承诺解决系统性种族歧视问题。\",\n",
    ")\n",
    "documents = [\n",
    "    document_1,\n",
    "    document_2,\n",
    "    document_3,\n",
    "    document_4,\n",
    "    document_5,\n",
    "    document_6,\n",
    "    document_7,\n",
    "    document_8,\n",
    "    document_9,\n",
    "    document_10,\n",
    "]\n",
    "\n",
    "# 将文档向量化，添加到向量数据库索引中，得到向量数据库对象\n",
    "db = FAISS.from_documents(documents, embeddings_model)"
   ],
   "id": "86256c836375dd1a",
   "outputs": [],
   "execution_count": 8
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 默认检索器使用相似性搜索",
   "id": "907103d8bdc78f61"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T02:44:54.297782Z",
     "start_time": "2025-11-26T02:44:53.986677Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# 获取检索器\n",
    "retriever = db.as_retriever(search_kwargs={\"k\": 4})  #这里设置返回的文档数\n",
    "source = retriever.invoke(\"经济政策\")\n",
    "for i, doc in enumerate(source):\n",
    "    print(f\"\\n结果 {i + 1}:\\n{doc.page_content}\\n\")"
   ],
   "id": "42372493220f1783",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "结果 1:\n",
      "外交政策：继续支持乌克兰对抗俄罗斯的侵略。\n",
      "\n",
      "\n",
      "结果 2:\n",
      "经济复苏：美国经济正在从疫情中强劲复苏，失业率降至历史低点。！\n",
      "\n",
      "\n",
      "结果 3:\n",
      "移民改革：提出全面的移民改革方案。\n",
      "\n",
      "\n",
      "结果 4:\n",
      "科技：增加对半导体产业的投资以减少对外国供应链的依赖。。\n",
      "\n"
     ]
    }
   ],
   "execution_count": 9
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### 分数阈值查询\n",
    "**只有相似度超过这个值才会召回**"
   ],
   "id": "51d600a96abfbdd8"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T02:44:54.696844Z",
     "start_time": "2025-11-26T02:44:54.305752Z"
    }
   },
   "cell_type": "code",
   "source": [
    "retriever = db.as_retriever(\n",
    "    search_type=\"similarity_score_threshold\",\n",
    "    search_kwargs={\"score_threshold\": 0.1}\n",
    ")\n",
    "source = retriever.invoke(\"经济政策\")\n",
    "for doc in source:\n",
    "    print(f\"📌 内容: {doc.page_content}\")"
   ],
   "id": "bb5ad0ea6e7c24e5",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "📌 内容: 外交政策：继续支持乌克兰对抗俄罗斯的侵略。\n",
      "📌 内容: 经济复苏：美国经济正在从疫情中强劲复苏，失业率降至历史低点。！\n",
      "📌 内容: 移民改革：提出全面的移民改革方案。\n",
      "📌 内容: 科技：增加对半导体产业的投资以减少对外国供应链的依赖。。\n"
     ]
    }
   ],
   "execution_count": 10
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T02:44:55.032922Z",
     "start_time": "2025-11-26T02:44:54.706846Z"
    }
   },
   "cell_type": "code",
   "source": [
    "docs_with_scores = db.similarity_search_with_relevance_scores(\"经济政策\")\n",
    "for doc, score in docs_with_scores:\n",
    "    print(f\"\\n相似度分数: {score:.4f}\")\n",
    "    print(f\"📌 内容: {doc.page_content}\")"
   ],
   "id": "30847495f74f49cd",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "相似度分数: 0.7347\n",
      "📌 内容: 外交政策：继续支持乌克兰对抗俄罗斯的侵略。\n",
      "\n",
      "相似度分数: 0.7330\n",
      "📌 内容: 经济复苏：美国经济正在从疫情中强劲复苏，失业率降至历史低点。！\n",
      "\n",
      "相似度分数: 0.7218\n",
      "📌 内容: 移民改革：提出全面的移民改革方案。\n",
      "\n",
      "相似度分数: 0.7218\n",
      "📌 内容: 科技：增加对半导体产业的投资以减少对外国供应链的依赖。。\n"
     ]
    }
   ],
   "execution_count": 11
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### MMR搜索",
   "id": "17aed2b5ea888ef1"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T02:44:55.432904Z",
     "start_time": "2025-11-26T02:44:55.038925Z"
    }
   },
   "cell_type": "code",
   "source": [
    "retriever = db.as_retriever(\n",
    "    search_type=\"mmr\"\n",
    ")\n",
    "source = retriever.invoke(\"经济政策\")\n",
    "print(len(source))\n",
    "for doc in source:\n",
    "    print(f\"📌 内容: {doc.page_content}\")"
   ],
   "id": "4066ee97335d7389",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n",
      "📌 内容: 外交政策：继续支持乌克兰对抗俄罗斯的侵略。\n",
      "📌 内容: 经济复苏：美国经济正在从疫情中强劲复苏，失业率降至历史低点。！\n",
      "📌 内容: 教育：提供免费的社区大学教育。。\n",
      "📌 内容: 科技：增加对半导体产业的投资以减少对外国供应链的依赖。。\n"
     ]
    }
   ],
   "execution_count": 12
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## 结合LLM",
   "id": "4ae129fe9a72a330"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 定义大模型",
   "id": "bf7b023f09047bec"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T03:00:39.084156Z",
     "start_time": "2025-11-26T03:00:39.058150Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import dotenv\n",
    "import os\n",
    "\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "dotenv.load_dotenv(override=True)\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\")\n",
    "os.environ[\"OPENAI_BASE_URL\"] = os.getenv(\"OPENAI_BASE_URL\")\n",
    "\n",
    "CHAT_MODEL = ChatOpenAI(\n",
    "    model=\"gpt-4o-mini\",\n",
    "    temperature=0\n",
    ")"
   ],
   "id": "fdc6aa5aa41375e",
   "outputs": [],
   "execution_count": 2
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 举例1：通过FAISS构建一个可搜索的向量索引数据库，并结合RAG技术让LLM去回答问题。",
   "id": "56fbfc31b2fe1798"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T02:44:57.783299Z",
     "start_time": "2025-11-26T02:44:55.472912Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain_community.document_loaders.text import TextLoader\n",
    "from langchain_text_splitters import CharacterTextSplitter\n",
    "from langchain_core.prompts import PromptTemplate\n",
    "from langchain_community.vectorstores.faiss import FAISS\n",
    "\n",
    "question = \"北京有什么著名的建筑？\"\n",
    "\n",
    "# 加载文档\n",
    "print(\"开始加载文档..........\")\n",
    "text_loader = TextLoader(\"asset/load/10-test_doc.txt\", encoding=\"utf-8\")\n",
    "source = text_loader.load()\n",
    "\n",
    "# 拆分文档\n",
    "print(\"开始拆分文档..........\")\n",
    "text_splitter = CharacterTextSplitter(\n",
    "    chunk_size=1000,\n",
    "    chunk_overlap=100\n",
    ")\n",
    "chunks = text_splitter.split_documents(source)\n",
    "print(len(chunks))\n",
    "print(chunks[0].metadata)\n",
    "for chunk in chunks:\n",
    "    print(f\">>>>>>>>>>>>>>>chunk.page_content:\\n{chunk.page_content}\")\n",
    "\n",
    "# 创建向量数据库\n",
    "print(\"开始创建向量数据库..........\")\n",
    "db = FAISS.from_documents(chunks, embeddings_model)\n",
    "print(\"开始检索文档..........\")\n",
    "retriever = db.as_retriever()\n",
    "docs = retriever.invoke(question)\n",
    "\n",
    "prompt_template = \"\"\"请使用以下提供的文本内容来回答问题。仅使用提供的文本信息，如果文本中\n",
    "没有相关信息，请回答\"抱歉，提供的文本中没有这个信息\"。\n",
    "\n",
    "文本内容：\n",
    "{context}\n",
    "\n",
    "问题：{question}\n",
    "\n",
    "回答：\n",
    "\"\"\"\n",
    "prompt = PromptTemplate.from_template(prompt_template)\n",
    "\n",
    "print(\"开始调用大模型..........\")\n",
    "chain = prompt | CHAT_MODEL\n",
    "resp = chain.invoke(input={\"question\": question, \"context\": docs})\n",
    "print(resp.content)"
   ],
   "id": "34c5f62dc411e3cb",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "开始加载文档..........\n",
      "开始拆分文档..........\n",
      "1\n",
      "{'source': 'asset/load/10-test_doc.txt'}\n",
      ">>>>>>>>>>>>>>>chunk.page_content:\n",
      "北京是中国的首都，拥有丰富的历史文化遗产和现代建筑。以下是一些著名的建筑：\n",
      "\n",
      "1. 故宫\n",
      "故宫，又称紫禁城，是明清两代的皇家宫殿，位于北京市中心。它是世界上现存规模最大、保存最完整的木质结构古建筑群之一，1987年被列为世界文化遗产。\n",
      "\n",
      "2. 天安门\n",
      "天安门位于故宫南端，是北京的标志性建筑之一。天安门广场是世界上最大的城市广场，可容纳100万人集会。\n",
      "\n",
      "3. 颐和园\n",
      "颐和园是清朝时期的皇家园林，以昆明湖和万寿山为基础，融合了江南园林的设计风格，1998年被列入世界文化遗产。\n",
      "\n",
      "4. 天坛\n",
      "天坛是明清两代皇帝祭天、祈谷的场所，其精巧的建筑结构和深厚的文化内涵使其成为世界文化遗产（1998年）。\n",
      "\n",
      "5. 长城（八达岭段）\n",
      "虽然长城横跨多个省市，但八达岭长城是最著名的北京段，被誉为\"世界第八大奇迹\"。\n",
      "\n",
      "6. 国家体育场（鸟巢）\n",
      "作为2008年奥运会主体育场，鸟巢以其独特的钢结构设计成为现代北京的地标。\n",
      "\n",
      "7. 中央电视台总部大楼\n",
      "俗称\"大裤衩\"，是现代北京最具争议也最具识别度的建筑之一。\n",
      "\n",
      "8. 国家大剧院\n",
      "位于天安门广场西侧，因其蛋壳造型被称为\"巨蛋\"，是世界最大的穹顶建筑之一。\n",
      "\n",
      "9. 北京大兴国际机场\n",
      "2019年投入使用的超大型国际航空枢纽，被誉为\"新世界七大奇迹\"之一。\n",
      "\n",
      "10. 鼓楼和钟楼\n",
      "位于北京中轴线北端，是元明清三代的报时中心，展现了古代中国的计时智慧。\n",
      "开始创建向量数据库..........\n",
      "开始检索文档..........\n",
      "开始调用大模型..........\n",
      "北京有以下著名的建筑：\n",
      "\n",
      "1. 故宫\n",
      "2. 天安门\n",
      "3. 颐和园\n",
      "4. 天坛\n",
      "5. 长城（八达岭段）\n",
      "6. 国家体育场（鸟巢）\n",
      "7. 中央电视台总部大楼\n",
      "8. 国家大剧院\n",
      "9. 北京大兴国际机场\n",
      "10. 鼓楼和钟楼\n"
     ]
    }
   ],
   "execution_count": 14
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 举例2：使用Chroma数据库",
   "id": "fcbf3ed33544d899"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-26T03:00:50.359269Z",
     "start_time": "2025-11-26T03:00:43.245697Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from langchain.text_splitter import MarkdownTextSplitter\n",
    "from langchain_community.vectorstores import Chroma\n",
    "from langchain.prompts import PromptTemplate\n",
    "from langchain_community.document_loaders import UnstructuredMarkdownLoader\n",
    "\n",
    "import shutil\n",
    "import os\n",
    "\n",
    "question = \"what is Chat Models?\"\n",
    "markdown_path = \"asset/load/11-langchain.md\"\n",
    "# 1.定义UnstructuredMarkdownLoader对象\n",
    "loader = UnstructuredMarkdownLoader(markdown_path)\n",
    "# 2.加载\n",
    "print(\"开始加载文档..........\")\n",
    "data = loader.load()\n",
    "splitter = MarkdownTextSplitter(chunk_size=1000, chunk_overlap=100)\n",
    "# 3.执行分割\n",
    "print(\"开始拆分文档..........\")\n",
    "documents = splitter.split_documents(data)\n",
    "print(len(documents))\n",
    "\n",
    "# 4. 向量数据存储（默认存储到内存中）\n",
    "print(\"开始创建向量数据库..........\")\n",
    "db = Chroma.from_documents(documents, embeddings_model)\n",
    "# 5. 向量检索\n",
    "print(\"开始检索文档..........\")\n",
    "retriever = db.as_retriever()\n",
    "docs = retriever.invoke(question)\n",
    "\n",
    "# 6.定义提示词模版\n",
    "print(\"开始定义提示词模版..........\")\n",
    "template = \"\"\"\n",
    "你是一个问答机器人。\n",
    "你的任务是根据下述给定的已知信息回答用户问题。\n",
    "确保你的回复完全依据下述已知信息。不要编造答案。\n",
    "如果下述已知信息不足以回答用户的问题，请直接回复\"我无法回答您的问题\"。\n",
    "\n",
    "已知信息:\n",
    "{context}\n",
    "\n",
    "用户问：\n",
    "{question}\n",
    "\n",
    "请用中文回答用户问题。\n",
    "\"\"\"\n",
    "# 7.得到提示词模版对象\n",
    "prompt_template = PromptTemplate.from_template(template=template)\n",
    "# 8.得到提示词对象\n",
    "prompt = prompt_template.invoke({\"question\":question,\"context\":docs})\n",
    "## 9. 调用LLM\n",
    "print(\"开始调用大模型..........\")\n",
    "response = CHAT_MODEL.invoke(prompt)\n",
    "print(response.content)"
   ],
   "id": "883e6c80d7c0ee9a",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "开始加载文档..........\n",
      "开始拆分文档..........\n",
      "13\n",
      "开始创建向量数据库..........\n",
      "开始检索文档..........\n",
      "开始定义提示词模版..........\n",
      "开始调用大模型..........\n",
      "聊天模型是新型的语言模型，它们接收消息作为输入并输出消息。有关如何从特定提供者开始使用聊天模型的详细信息，请参阅支持的集成。\n"
     ]
    }
   ],
   "execution_count": 3
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
